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ABSTRACT 

Introduction: Activated protein C (aPC) plays a pivotal 
role in modulating a severe inflammatory response and 
is thought to be beneficial for patients with sepsis. 
However, several meta-analyses of randomised 
controlled trials (RCTs) show that aPC is not 
significantly associated with improved survival in 
critically ill patients with sepsis. One suggestion is that 
these analyses simply ignored observational evidence. 
The present study aims to quantitatively demonstrate 
how observational data can alter the findings derived 
from synthesised evidence from RCTs by using a 
Bayesian approach. 

Methods and analysis: RCTs and observational 
studies investigating the effect of aPC on mortality 
outcome in critically ill patients with sepsis will be 
included. The quality of included RCTs will be 
assessed by using the Delphi list. Publication bias will 
be quantitatively analysed by using the traditional 
Egger regression test and the Begg rank correlation 
test. Observational data will be used as the informative 
prior for the distribution of OR. A power transformation 
of the observational data likelihood will be considered. 
Observational evidence will be down-weighted by a 
power of a which takes values from 0 to 1 . Trial 
sequential analysis will be performed to quantify the 
reliability of data in meta-analysis adjusting significance 
levels for sparse data and multiple testing on 
accumulating trials. 

Trial registration number: PROSPERO 
(CRD42014009562). 
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INTRODUCTION 

Sepsis is defined as systematic inflammatory 
response syndrome (SIRS) caused by infec- 
tion. 1 Levels of severity vary widely depend- 
ing on the presence of shock and organ 
failure. Sepsis is a leading cause of morbidity 
and mortality in intensive care units. In the 
USA alone, there were over 750 000 esti- 
mated cases in 1995, 2 and sepsis accounts for 



over 25% of admissions to ICUs in Europe. 3 
Due to its significant impact on global 
health, every effort has been made to 
improve the survival of patients with sepsis. 
One such initiative is the Surviving Sepsis 
Campaign (SSC) with the objective of redu- 
cing mortality from sepsis by 25%. 4 Various 
strategies have been implemented to achieve 
this aim, such as early goal directed therapy, 
early use of broad spectrum antibiotics, 
source control and low tidal volume ventila- 
tion. Although the sepsis mortality rate has 
subsequently declined, the SSC goal is far 
from being achieved. 5 

Activated protein C (aPC) has pleiotropic 
biological effects and plays a pivotal role in 
modulating the severe inflammatory 
response which occurs in sepsis. Its biological 
effects include, but are not limited to, reduc- 
tion of thrombin production by inactivating 
factors Va and VIII, and inhibition of IL-1, 
IL-6 and TNF-a production by monocytes. 6 
Many observational studies (OS) have shown 
significantly improved survival outcomes in 
patients with sepsis treated with aPC com- 
pared with controls. Furthermore, these 
encouraging results have been confirmed in 
the milestone clinical trial PROWESS. 
However, the findings have not been repli- 
cated in subsequent randomised clinical 
trials, and thus enthusiasm for aPC has 
declined. 

Randomised controlled trials (RCT) are 
designed to test the biological efficacy of a 
particular treatment, while observational 
studies test the effectiveness of that treatment 
in the real world setting. 7 Differences in effi- 
cacy and effectiveness may result from issues 
related to trial design, patient selection and 
therapeutic implementation. Some systematic 
reviews exploring the effect of aPC on sepsis 
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exclusively focused on RCTs while ignoring evidence 
from OS, and consistently showed that aPC had a 
neutral effect on survival outcomes. 8 9 We propose that 
although RCTs are the 'gold standard' for the definite 
determination of the clinical efficacy of an intervention, 
OS cannot simply be ignored in evidence synthesis. Kalil 
and LaRosa provided a frequentist analysis of both obser- 
vational and randomised studies, but no Bayesian ana- 
lyses were performed. 10 From the Bayesian perspective, 
OS can be incorporated into the analysis and an inform- 
ative prior distribution on the treatment effect derived 
from the observational data. 11 In contrast to previous 
meta-analysis, we will incorporate observational data into 
analysis using the Bayesian approach. Furthermore, add- 
itional RCTs will be incorporated in order to update the 
systematic review. 

METHODS 
Search strategy 

We will search electronic databases including the 
Cochrane Central Register of Controlled Trials 
(CENTRAL), PubMed, EBSCO, EMBASE and ISI Web 
of Science from inception to January 2014. Our core 
search consists of terms related to aPC and sepsis (see 
table 1 for the detailed search strategy to be used in 
PubMed). Strategies will be adapted to other databases. 
There will be no language restriction. The references of 
systematic reviews will be reviewed to identify additional 
eligible articles. 

Studies to be included 

We will include RCTs and OS for analysis. OS will 
include: (1) cohort studies using multivariable analysis 
with aPC treatment as one of the covariates; (2) cohort 
studies using propensity analysis; (3) case-control 
studies; (4) studies with both prospective and retrospect- 
ive designs; and (5) all OS irrespective of their methodo- 
logical design quality. 



Table 1 


Search strategy performed in PubMed 






Number of 


Items 


Search terms 


citations 


1# 


((activated protein C[Title/ 


4460 




Abstract]) OR xigris[Title/ 






Abstract]) OR drotrecogin alfa 






[Title/Abstract] 




2# 


(sepsis[Title/Abstract]) OR 


72 635 




septic shock[Title/Abstract] 




3# 


(((mortality[Title/Abstract]) OR 


875 580 




safety[Title/Abstract]) OR 






adverse events[Title/ 






Abstract]) OR bleeding[Title/ 






Abstract] 




1# AND 




531 


2# AND 






3# 









Studies to be excluded 

We will exclude studies that: (1) do not report mortality 
as an endpoint; (2) are a secondary analysis of a primary 
study whose data have been published elsewhere; and 
(3) only include a single arm so that no comparison can 
be made between different treatment strategies (eg, 
such as analysis of risk factors) . 

Data extraction 

A custom-made form will be used to extract the follow- 
ing data from eligible studies: name of the first author, 
year of publication, sample size, illness severity scores 
(APACHE II, SOFA and SAPS), number of deaths in 
each arm, total number of participants in each arm, 
bleeding or haemorrhage events in each arm, OR of 
treatment versus non-treatment for mortality, the 
method used for covariate adjustment (propensity score 
analysis, logistic regression model) and the design of the 
OS (prospective vs retrospective). The adverse event of 
bleeding will be divided into two categories: major 
bleeding (terms consist of combinations of 'massive', 
'major' and 'bleeding', 'haemorrhage') and any bleed- 
ing (terms consist of combinations of 'minor' and 
'bleeding', 'haemorrhage'). If only the risk ratio (RR) is 
reported, we will transform it into the OR by using 
standard formula (described elsewhere 12 ): 

RRx(l-CER) 
1 - CER x RR 

where CER indicates control event rate (same as control 
group risk). Mortality is defined variably across studies 
(eg, 28-day, in-hospital, 60-day or 90-day) and we will 
include all types of definitions for analysis. 

Quality assessment of RCTs and OS 

Quality assessment of included RCTs will be performed 
by using the Delphi list, which consists of nine items: 
sequence generation, allocation concealment, baseline 
characteristics, eligibility criteria, blindness to outcome 
assessor, blindness to care provider, blindness to patient, 
use of point estimate and variability for outcome mea- 
sures, and use of intention to treat analysis. 13 The explan- 
ation and rating for each item are given in table 2. 
Quality assessment of OS will be performed by using the 
modified Newcastle-Ottawa scale which has been 
described elsewhere (table 3). 14 

Publication bias 

Publication bias will be quantitatively analysed by using 
the traditional Egger regression test and Begg rank cor- 
relation test. 15 16 The Begg rank correlation test investi- 
gates the relationship between the standardised OR and 
sample size or variance by using the Spearman rank cor- 
relation. 17 In the Egger regression test, the standard 
normal deviate (the OR divided by its SE) is regressed 
against the estimates precision. The intercept of the 
regression line is an estimate of asymmetry: the larger its 
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Table 2 Quality assessment of randomised controlled trials using tools adapted from the Delphi list 



Items 



Explanation 



Rating 



Sequence generation 
Allocation concealment 

Baseline characteristics 

Eligibility criteria 
Blindness to outcome 
assessor 

Blindness to care provider 
Blindness to patient 
Point estimate and 
variability 
Intention-to-treat 



Is the method of sequence generation clearly reported? Yes/no/unclear 

Is treatment allocation concealment (using an opaque envelope, central Yes/no/unclear 
allocation) performed? 

Are the groups similar at baseline regarding the most important prognostic Yes/no/unclear 
factors? 

Are eligibility criteria clearly specified? Yes/no/unclear 

Is the outcome (mortality) assessor blinded? Yes/no/unclear 

Is the allocation unknown to the treating physician? Yes/no/unclear 

Is the patient blinded? Yes/no/unclear 

Are the point estimate and variability reported for the outcome measure? Yes/no/unclear 

Does the analysis include intention to treat analysis? Yes/no/unclear 



deviation from origin, the more significant the asym- 
metry. 18 A contour enhanced funnel plot will be used to 
visually assess the presence of publication bias. OR is 
plotted on the horizontal axis, and precision is plotted 
on the vertical axis, with asymmetric distribution of com- 
ponent studies representing potential publication bias. 
Contour lines are added to the plot at conventional 
statistical significance levels of <0.01, <0.05 and <0.1. 
A funnel contour enhanced plot can aid interpretation 
of the funnel plot. If studies are missing in the non- 
significance area, it is likely that the asymmetry is caused 
by publication bias. Conversely, if studies are in the sig- 
nificance area, the asymmetry is more likely caused by 
factors other than publication bias, such as study 
quality. 19 

Sensitivity or subgroup analysis 

Sensitivity analysis will be performed by excluding 
studies with poor methodological design. Subgroup ana- 
lysis will be performed to explore confounding factors 



such as shock versus non-shock, and the effect of aPC 
modified by disease severity. If there are enough studies 
with the same definition of mortality (n>5), subgroup 
analysis will be performed by different mortality 
definitions. 

Statistical analysis 

Three key components of Bayesian analysis are prior, likeli- 
hood and posterior. The quantity of interest in our study is 
the OR for mortality. Observational data are used as the 
informative prior for the distribution of OR. For studies 
using a logistic regression model for risk adjustment, we 
will extract adjusted OR and relevant 95% CI for analysis. 
For studies using propensity matched analysis, the OR 
from matched samples are calculated. Random effects 
meta-analysis will be performed to combine the results 
obtained from OS, by using a Bayesian approach. 20 The 
WinBUGS code for performing the calculation is shown in 
table 3. The pooled OR will be transformed by natural log 
to In (OR) to improve normality. The SE in the natural log 



Table 3 Quality assessment of included observational studies using the modified Newcastle-Ottawa scale 



Selection Representativeness of the exposed 
cohort 

Outcome of interest is not present at 

the start of the study 
Comparability Comparability of cohorts on the basis 

of design or analysis 
Outcome Assessment of outcome 



Selection of the non-exposed cohort 
Ascertainment of exposure 



Is follow-up long enough for outcome 
to occur? 

Adequacy of follow-up of the cohort 



This item will be assigned a when all eligible patients with severe 
sepsis or septic shock are included in the analysis during the study 
period 

This item will be assigned a when all eligible patients without 
aPC treatment are included in the analysis during the study period 
This item will be assigned a 'V when aPC administration is directly 
obtained from a medical chart, not from reporting by the patient 
This item will be assigned a when the subject is alive at the time 
of enrolment 

Baseline characteristics of aPC and control groups are comparable. 
Usually this can be found in table 1 of the original article. 
This item will be assigned a 'Vwhen mortality is assessed by the 
investigator, not by the report of the patient's family or next-of-kin 
Adequate follow-up is carried out during hospital stay, ICU stay or 
redefined study time 

This item will be assigned a when the follow-up rate is >80% 



aPC, activated protein C. 
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scale can be transformed from the 95% credible interval 
by using the equation: 

Standard error (a) = _^E !^ 

v ; 2 x 1.96 

where L up and L lo represent the upper and lower limits of 
the 95% credible interval. The precision is the reciprocal 
ofSE. 

The framework to incorporate observational data as 
informative prior is presented by Chen and Ibrahim. 11 
Model development has been described elsewhere but 
we repeat it here for the reader's benefit. Let the data 
from RCTs be denoted by D, and the likelihood of RCTs 
be denoted by L(0|D). Suppose we have data from OS 
which are denoted by D 0 . Furthermore, let P(6) denote 
the prior distribution for 9 before OS are incorporated. 
P(0) is the initial prior distribution for 6. Given a, the 
power prior distribution of 0 is defined as: 

P(0|D o ,a)ocL(e|Do) a xP(e|c o ) 

where c 0 is the hyperparameter for initial prior, and a is 
used to weight observational evidence relative to the like- 
lihood of RCT evidence. The value of a controls the 
impact of observational evidence on P(0|D o , a). When 
evidence from RCTs is added to the model, a power 
transformation of the observational data likelihood is 
considered: 

P(8|Data) = L(0|RCTs) x [L(0|Obs)] a x P(8) 

where P(0|Data) is the posterior distribution for model 
quantities, [L(0|Obs)] is the likelihood function derived 
from observational data, and L(0|RCTs) is the likelihood 
function from RCT data. The weight of observational data 
is counted by the power a. The power takes values from 0 
to 1 . If a=0, the observational data are essentially removed 
from analysis and only RCTs are used for evidence synthe- 
sis; if a=l, observational data are taken at their 'face value' 
and not discounted at all. Traditional meta-analyses such 
as those done in The Cochrane Collaboration included 
only RCTs that actually render a=0. In our analysis, a will 
take 12 values ranging between 0 and 1 (0.000001, 0.001, 
0.1, 0.2, 0.3, 0.4, 0.5, 0.6, 0.7, 0.8, 0.9, 1.0), resulting in a 
series of posterior distributions for OR. As shown in 
table 4, the WinBUGS code is composed of three parts. 
Part (1) is to repeat meta-analysis of RCTs 12 times, once 
for each value of a to discount the observational evidence. 
Part (2) is the meta-analysis model. In this section, i repre- 
sents the component studies and k indices each of the 12 
meta-analyses. These meta-analyses differ from each other 
only in the prior distribution for the overall pooled 
effect d, which is represented by: 

d[k] - dnorm(0.33,prec.d[k]). 

The mean of prior distribution (the figure 0.33 in the 
expression is used for illustration purposes, and is not 
obtained from real analysis) is the natural log of the pooled 



OR (LOR) estimated from observational data. The pooled 
OR is estimated with a Bayesian approach with a random 
effects model. The code for the random effects 
meta-analysis is shown in table 4. The precision of the prior 
distribution, prec.d[k], is determined in part (3). Part (3) is 
to calculate precision of the prior discounted by using a. 21 

Convergence diagnostics will be explored by running 
two chains. Simulated values will be compared to iden- 
tify when they become similar. History plots with differ- 
ent chains superimposed (in different colours) will help 
to determine convergence. Furthermore, we will use the 
Brooks-Gelman-Rubin diagnostic to test convergence. 
The procedure will produce three coloured lines (red, 
blue and green) . Convergence is deemed to occur when 
the red line settles close to 1 and the blue and green 
lines converge together. 

Trial sequential analysis (TSA) is performed to quan- 
tify the reliability of data in meta-analysis adjusting sig- 
nificance levels for sparse data and multiple testing on 
accumulating trials. 22 Trial sequential monitoring 
boundaries are used to control the risks for type I and II 
errors and to indicate whether additional trials are 
needed. A zero-event trial will be handled by the con- 
stant continuity correction method with a correction 
factor of 0.5, that is, 0.5 is added to each cell of the 2x2 
table. 23 The information size calculation requires the 
mortality rate in the control group and the minimal 
effect size for the intervention. We predefined that the 
mortality in the control group is 30%, and the interven- 
tion is able to reduce the relative risk by 10%. The con- 
ventional a and (3 are 0.05 and 0.2, respectively. 
Meta-analysis will be updated by adding component 
studies sequentially in the order of publication. 

Statistical analysis will be performed by using 
WinBUGS (Imperial College and MRC, UK) and Stata 
V.12.0 (College Station, Texas, USA). TSA will be per- 
formed by using the software TSA V.0.9 Beta 
(Copenhagen Trial Unit, 2011). 

Results to be reported 

Search results will be displayed in a flowchart. Pooled 
results from conventional meta-analysis techniques will 
be displayed in forest plots separately for RCTs and OS. 
Publication bias as shown in funnel plots will also be dis- 
played, again separately for RCTs and OS. The results of 
TSA will be reported graphically. Random effects 
meta-analysis using a Bayesian approach will be used to 
pool summary effects for observational evidence and the 
results will be reported by using a caterpillar plot. 
Summary OR will also be plotted against different values 
of a to examine how observational evidence influences 
the summary effect. The Brooks-Gelman-Rubin plot 
will be used to display convergence diagnostics. 

DISCUSSION 

aPC was once the only approved drug for the treatment 
of sepsis. However, it was withdrawn from the market 
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Table 4 WinBUGS codes for performing random effects meta-analysis and meta-analysis incorporating observational data 



Random effects meta-analysis 



Informative prior with observational data 



Modelt 



Data* 



lnitials§ 



model { 
for (i in 1:N) 
{ 

P[i]<-1/V[i] Y[i]~dnorm(delta[i], P[i]) delta[i]~dnorm(d, prec) 
OR[i]<-exp(delta[i]) 

} d~dnorm(0, 1.0E-5) OR[13]<-exp(d) tau~dunif(0,10) tau. 

sq<-tau*tau prec<-1/tau.sq 

} 



list(Y=c(-0.51083, -0.73397, -0.24846, -0.15082, -0.54473, 
-0.52763, -0.36817, -0.13926, -0.75502, -0.27444, 
-0.26136), 

V=c(1. 706611, 0.01954, 0.035483, 0.021832, 0.010326, 

0.033478, 0.011817, 0.005765, 0.089499, 0.004559, 

0.022782), 

N=11) 

list( 

d = 472.0235128342391, 
delta = c( 

470.6994400270435, 472.3980455275865, 
472.201 1 37881 263, 472.01 98057372273, 
471 .8605396435204, 

470.2850099832592, 469.582973561 8464, 

473.0258057826344, 470.39322381 4331 6, 

469.5792223324207, 

469.6419041364815), 

tau = 0.8303798133648838) 



model { 

# (1) create multiple datasets 

for (i in 1:5) { 
for (k in 1:12) { 

rc[i, k]<- rc.dat[i] rt[i, k]<-rt.dat[i] nc[i, k]<-nc.dat[i] nt[i 

k]<-nt.dat[i] } 

} 

# (2) estimate RCT meta-analysis model for each 
value of data 

for (kin 1:12) { 
for (i in 1:5) { 
rc[i,k]~dbin(pc[i,k], nc[i,k]) 
rt[i,k]~dbin(pt[i,k], nt[i,k]) logit(pc[i,k])<-mu[i,k] 
logit(pt[i,k])<-mu[i,k]+delta[i,k] 
mu[i,k]~dnorm(0.0, 1.0E-6) 
delta[i,k]~dnorm(d[k], prec[k]) 
or[i,k]<-exp(delta[i,k]) 
} 

d[k]~dnorm(0.33, prec.d[k]) 
OR[k]<-exp(d[k]) 
prec[k]<-1/tau.sq[k] 
tau.sq[k]<-tau[k]*tau[k] 
tau[k]~dunif(0,5) 
} 

# (3) calculate precision of prior (from 
meta-analysis of obs studies) downweighted 
using alpha 

for (kin 1:12) { 
prec.d[k]<-alpha[k]*271 .3 
} 
} 

list(rt.dat=c(0,2,3,2,3), 
nt.dat=c(67,45,34,56,34), 
rc.dat=c(2,3,4,2,0), 
nc.dat=c(44,56,78,1 23,35), 
alpha=c(0.0001, 0.2, 0.8) 



list(d = c(0,0,0), 

delta = structure(.Data = c(**place 5*12=60 initial 
values here**), 
.Dim = c(5,12)), 

mu = structure(.Data = c(**place 5*12=60 initial 
values here**), 
.Dim = c(5,12)), 
tau = c(1, 1,1, 1,1, 1,1, 1,1, 1,1,1) 



list( 
d = 0, 

delta = c(0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0), 
tau = 1) 



tContents following # are not syntax used for analysis, but are used to annotate corresponding codes. 
JData are used for illustration purpose and are not obtained from the real analysis. 
§lnitial values are randomly generated and do not represent the actual values used in analysis. 
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after the large clinical trial PROWESS-SHOCK failed to 
identify any beneficial effect in patients with sepsis. 
However, in the first place, aPC was approved for use in 
patients with sepsis because the PROWESS study demon- 
strated a significant beneficial effect, with the study 
being stopped early because of its efficacy. 24 
Furthermore, a large number of OS also showed a large 
beneficial effect with the use of aPC. Clinicians may be 
confused by these seemingly differing results. It is still 
largely unknown whether aPC is beneficial for specific 
subgroups of patients with sepsis. In this situation, the 
synthesis of evidence for decision making may help to 
address these conflicting findings. As a result, a few 
study groups have conducted systematic reviews and 
meta-analyses to provide comprehensive and up-to-date 
evidence for clinical use. The Cochrane Collaboration 
has also published the results of an updated 
meta-analysis on the effectiveness of aPC for sepsis, 
which however showed a neutral effect. 8 However, this 
meta-analysis only included RCTs. There is no doubt 
that the RCT is the gold standard for supplying evidence 
for medical decision making and can provide high level 
evidence on the comparative effectiveness of interven- 
tions. However, there are some circumstances where 
non-randomised evidence should be incorporated in 
order to estimate effectiveness. These include situations 
where there are concerns about internal and external 
validity (only effective in specialised centres or highly 
selected subjects) and size (estimates are imprecision). 
Many RCTs in critically ill patients showed a neutral 
effect of the intervention under investigation. In other 
situations, initial trials showed a beneficial effect of the 
intervention which, however, was refuted by a subse- 
quent meta-trial. Reasons for these negative results 
include timing of enrolment, endpoint selection and 
heterogeneous subjects. 25 26 

When both RCTs and OS are available, common prac- 
tice is to combine data by equally weighting these two 
types of studies. When evaluating protective ventilation 
for non-acute respiratory distress syndrome (ARDS) 
patients, Serpa Neto et af 1 combined both RCTs and 
observational data with equal weights. The use of such a 
practice is partly due to difficulties in model building 
under the conventional statistical framework. However, 
there will be more flexibility for model building under 
the framework of a Bayesian perspective. The advantages 
of Bayesian analysis include but are not limited to: (1) it 
allows for evidence derived from a variety of sources 
including RCTs and observational data; (2) it enables a 
direct probability statement regarding the quantity of 
interest; and (3) all parameter uncertainties can be 
automatically accounted for. 28 We believe that the 
present study will provide new evidence for the effective- 
ness of aPC on mortality in patients with sepsis. 
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